1. Field of the Invention
This invention deals generally with an algorithm for increasing the spatial acuity of Focal Plane Array based Electro-Optic imaging systems by accumulating multiple frames of imagery into a single composite image and thus reducing the effective focal length of a viewing lens.
2. Description of the Related Prior Art
Single frame digital image restoration is a widely implemented mathematical technique that can compensate for known or estimated distortions endemic to a given digital image, improving the perceptual acuity and operational resolution of the constituent digital imaging sensor. (See Chapter 8 of Fundamentals of Digital Image Processing, A. K. Jain, Prentice Hall 1989)
The performance of such single-frame restoration techniques can be bounded by two limitations:                1) Insufficient spatial sampling of the projected optical image when measured by a single-frame capture of the projected optical image by a focal plane array. Depending on the F-number of the lens and the physical spacing (pixel pitch) of detectors, this situation may result in spatial alias distortion that is unrecoverable in a general sense.        2) Noise of the constituent pixel detectors in a focal plane array, and the associated read-out electronic microcircuits, which will limit the performance of any subsequent restoration filter.        
When imaging an object of interest, a sensor may often stare at that object for sufficient time to create a video sequence of images that dwell, with the possibility to drift, over the particular object. For many applications, only a single frame is recorded and processed, discarding the statistically innovative information that may be contained in additional, but unexamined images captured by the focal plane array.
Straightforward implementation of resolution enhancement through multiple frames of imagery have been implemented by controlled micro-dither scanning of a sensor (W. F. O'Neal “Experimental Performance of a Dither-Scanned InSb Array” Proceedings on the 1993 Meeting of the IRIS Specialty Group on Passive Sensors), where a stationary scene is imaged by a sensor subject to a well controlled pattern of orientation displacements, such as an integer fraction of a pixel. Image recovery is then implemented by appropriately interlacing the constituent images into a composite image with an integer-multiple increase in sampling density. Such techniques are very effective in suppressing alias distortions of any single frame, but may come at the cost of stabilization requirements that limit their implementation in practical, man-portable sensor systems.
Without any deliberate dithering, such video sequences of images may still be subject to unknown displacements, which can be exploited to provide the same benefits as controlled dither. There has been a history of research in algorithms to implement a multi-frame image restoration on such data sets (T. S. Huang., “Multiple frame image restoration and registration,” in Advances in Computer Vision and Image Processing, vol. 1, JAI Press, 1984.). The preponderance of these algorithms follows a common, non-linear approach to this problem:                1) Pre-suppose the existence of a high-resolution image, perhaps sampled at some integer multiple of the number of pixels of the constituent images. Seed this high resolution image with some initial guess, such as the interpolation of any single frame to the higher spatial sampling rate.        2) Derive some guess of the motion of the video sequence relative to the high resolution image. Displace and down-sample the high resolution image so as to create a synthetic video sequence consistent with the observed video sequence.        3) Determine some form of error between the synthetic and actual video sequence.        4) Adjust the estimates of both the high-resolution image and the scene motion so as to reduce the error between synthetic and actual video sequences.        5) Repeat steps 3 & 4 until a convergence in error has been reached.        
This approach to multi-frame image restoration is plagued by three limitations                1) Iterative algorithms often exhibit long convergence times and are computationally intense.        2) Numerical techniques for adjusting the estimates of step 4 often depend on specifying an underlying probability distribution model. Such Maximum Likelihood or Maximum A-Postori techniques prove to be numerically unstable if the underlying data deviates from such idealized statistical models.        3) Many such algorithms are constrained to cases of simple motion models, such as uniform displacements between frames of video, which may not fully represent the true motion of the sequence.        4) Final restoration of the high resolution image additionally depends on an empirical smoothing kernel with little or no analytic derivation.        